making your own engine: a survey of the landscape

20220714

i have been meaning to write something like this for a while. this is probably only the beginning of a series of articles i will be writing about the practical, technical aspects of making your own game engine.

why? because unity just “merged” with a company that makes malware. many people are jumping ship to unreal engine. i think this is a bad idea.

a lot of the justifications for “why” are covered in this fantastic blog post by tyler glaiel. i have my own further justifications for why you should consider making your own engine, but mostly i think that in light of recent decisions by unity and past decisions by epic games, i think the motivation of independence that he talks about is extremely important.

if you think an open source engine like godot is a solution to this problem: it’s not. as an avid user of mastodon, i can tell you first-hand that “open source” doesn’t solve all your problems. the people in charge of open source projects can and do make awful decisions all the time, and then the onus is on you to provide the free labor to fix the problems they created if you can’t convince them that they’ve made a mistake.

i firmly believe that when you rely on other people’s code, it should be as simple as possible, and do as little as possible, so you can replace it in the future easily if need be. this will never be possible with giant do-it-all engines.

in particular, i think that any engine with a built-in editor is doomed to failure. the problem with built-in editors and asset pipelines is that these are the things that need to be the most specialized between projects. if you’re coming from unity, every large unity project winds up looking different, and you lose the ostensible benefits of standardization.

the process of making a game is largely the process of figuring out what tools you need to make the game, and then making those tools. how do your artists like to work? what are the resolution of your images? what other tools are in your pipeline? built-in editors-as-products fail because they are trying to account for every possible workflow and in the end they can account for none of them.

would you rather bend your workflow to accommodate unity, unreal, or godot? or would you rather build tools that allow you and your team to work how you want?

i don’t think there’s zero use cases for these sorts of engines. you might be in a situation where you need a lot of the features these engines provide, and you don’t have the expertise to do it yourself. in particular, if you are making a very complex 3D game with a lot of skeletal animation and high fidelity graphics, it might make sense to use unreal. for what it’s worth, though, the studio i work at is having great success with a custom engine on our 3D game with a lot of skeletal animation and high fidelity graphics.

in the end, i think making your own engine is something you should consider as a realistic possibility for making the things you want to make faster and with more control over the final product. i don’t like that the entire indie games industry is caught in the death grip of two megacorporations. i especially don’t like that talking about making your own engine is met with mockery and derision. it’s a legitimate choice that you can make, and that lots of people made for a long time. the current status quo has not always been this way: indie games in the 00s were very often made with custom engines!

what i want to emphasize more than anything is that a better world is possible. there is no such thing as a perfect tool, but we can be doing better. i don’t think that making your own engine is reserved for experts or weirdos. i think that you, a person who knows how to program video games with unity, can become the sort of person who can make their own engine. and i know this because i did it. i used to think that i could never make a game without unity. i was wrong, and doing so was so much better than i could possibly have imagined.

so: here is a survey of the options available to you if you want to take the plunge into taking control of your development process. they are organized by programming language, in order of how familiar and friendly someone jumping ship from unity would find them. these are the options i personally would consider. there are others available, you can make a game with basically anything, but based on my experience these are the most realistic, practical choices.

luxe

a bit of a mea culpa: i originally misunderstood luxe. i have been in the closed beta for luxe since last august, and i tried it out a little bit over a weekend and was unimpressed. i came away with some incorrect assumptions about the engine, and for that i apologize to ruby, and the whole luxe community. the truth is, luxe is awesome. it’s a lot like love2d, honestly, but it uses a substantially less busted programming language called wren. i still have gripes about wren (i wish it were statically typed), but it’s way, way less weird than lua.

the luxe community is excellent, too. the engine is still in closed beta, but getting a beta invite is pretty easy. if using unstable, beta software doesn’t put you off, definitely look into it. hopefully progress towards stable release is made quickly and everyone can have access to this incredible framework!

c#

C# is the programming language that unity uses, and it needs no real introduction. i have some quibbles with it that eventually pushed me away from it, however.

first, its build system is awful. unity hides a lot of the worst aspects of c# as a language, and you’ll be dealing with the full brunt of those issues if you choose c# for your custom engine. It is a weird fiddly nightmare to get a project with multiple dependencies working. if you go this route, i recommend making extensive use of git submodules rather than using the nuget package manager.

second, as a virtual machine language, it is not very portable. the only way to get it running on a console is to transpile it to C++. the main option for this is BRUTE, but it is by no means a painless process. if you don’t plan on porting to consoles, or you have the resources to hire a professional porter like ethan lee (and if you’re targeting consoles, you probably do), this may not be an issue. but still, it’s there.

UPDATE: upon this post going live, i was told by one of the FNA maintainers, caleb cornett, that the c# porting situation has improved! microsoft has developed technology known as “NativeAOT,” which allows you to ahead-of-time compile .NET code to native code on your target platform, which the FNA maintainers already have working on switch and xbox. i don’t know how much of a pain in the ass this is, based on my experience with microsoft products i would guess “still kind of a pain in the ass,” but you don’t lose as much performance using nativeaot as you would with brute.

FNA

speaking of ethan lee, his main project is FNA, a reimplementation of microsoft’s deprecated XNA game framework. XNA was used to make a huge number of indie games in the late 00s and early 10s, and FNA was created as a preservation project to make porting those games to new consoles easier.

this means that FNA is battle-tested and extremely stable. if you are dead-set on using c# as your language, FNA is what i would recommend. its API is simple and friendly, it does the hard stuff and gets out of the way. its community is also fantastic, and includes some of the very best programmers in the games industry.

it’s not without its issues, the main one being that it is a preservation project first and foremost. making new games with FNA is supported, but they will never make breaking changes to the API. on the one hand, this is good; your project will never stop working because of something the maintainers did. on the other hand, it makes updating it to support more modern features like the vulkan graphics API more difficult, and sacrifices performance for preserving the API in amber.

moonworks

speaking of the vulkan graphics API, one of the programmers responsible for vulkan support in FNA, evan hemsley, took his experience as an FNA maintainer and used it to make moonworks, a framework that pitches itself as a modern take on the XNA concept.

moonworks is extremely promising, and something to keep your eye on, but i wouldn’t recommend it right now. it’s being used in the development of samurai gunn 2, but the full port of that game from game maker to moonworks is not finished yet. the moonworks API is extremely unstable right now. i was working on a moonworks tutorial, but put it aside until the API settles down a bit.

still, the discord is a fantastic place to keep up with active development, and evan has made a variety of additional tools that will be useful to you making a game in c# even if you’re using fna.

moontools

speaking of additional tools, “moontools” here refers to all of the wonderful c# tools evan has made, which are available on his gitea. the most significant of these is moontools.ecs, an entity component system for C#. i have a future article in the works about why i think ecs is the best way to make video games, but for now, suffice it to say that i do think that. moontools.ecs is an extremely simple, easy to use ecs that will make your experience of programming a video game approximately one million times easier and less frustrating.

the best feature of moontools.ecs is its concept of “messages,” which allow you to easily communicate data between systems. every ecs should have this feature.

haxe

haxe is a surprisingly cool language that i’ve actually written a lot of code in. it’s a language heavily inspired by actionscript, so if you ever programmed flash games in the past you’ll feel right at home. unlike actionscript, haxe’s big selling point is that it compiles to a huge number of targets.

haxe can compile to javascript for deployment on the web, it can compile to its own virtual machine for deployment on desktop platforms, and it can compile to c++ for deployment on consoles. its build system is much friendlier than C#’s.

haxe shares the same issues with C# in that it’s garbage collected, which can cause performance issues if not handled carefully.

haxe’s biggest flaw is that it’s not in very common use. a lot of flash game developers used it to make games once flash stopped being supported in many places, but many of those people have jumped ship to unity. this can change, though! i think haxe is a great choice. some games that have been made in haxe that you might not expect include a good snowman is hard to build, papers, please, rymdkapsel, even the ocean, and dead cells.

heaps

heaps is probably the most robust and flexible game framework written in haxe. it’s what they used to make dead cells. it features robust tools for both 2d and 3d game development. i have written a 3d renderer in heaps, and i had a great time. honestly, i don’t have anything to complain about, heaps is great.

openfl

openfl is a complete reimplementation of the adobe flash API in haxe. it’s a lot like FNA, in that respect. it’s often used for porting old flash games to modern systems, but it’s useful for new projects too. i’d probably choose heaps over it, but if you have experience with the flash API from back in the day, you might like this.

haxeflixel

haxeflixel is a reimplementation of the flixel flash game framework, but based on openfl. it provides some nice abstractions on the raw flash API to make programming some things easier. i’ve also used this before, and i think it’s fine. honestly, all of these choices are pretty good.

lua

lua is a very common language in game development. it’s an embedded language, which means that it runs within the c/c++ runtime. this makes it highly portable: if c/c++ can run on a platform, lua can run there too. it’s also very friendly to people who might find programming a little scary or stressful.

it has its drawbacks. it’s dynamically typed, which makes it less scary to newbies but can cause problems with complex projects. static type systems can catch lots of errors at compile time that dynamic type systems can’t. it also indexes arrays from 1 instead of 0, which is a war crime as far as i’m concerned. in general, lua is just kind of a weird language, which stems from its main goal of being as small as possible.

despite all of this, i still think lua is a good ecosystem, largely because there are a number of languages that compile to lua that fix a lot of its weird quirks.

löve

if you’re making a game with lua, you’re going to be doing so with love2d. honestly, love2d rules. i can’t think of a bad thing to say about it. despite the 2d in its name, you can actually make 3d games with it (we’ll get there in a bit). the phenomenal excellence of love2d is the main reason i am comfortable recommending lua as the best choice for people jumping ship from unity despite lua’s shortcomings. it has a great, active community, years of development, and tons of great features.

the only thing the love2d ecosystem lacks, in my opinion, is a definitive ecs framework. fortunately, ecs is so conceptually simple you can easily build your own, and i will be covering how to do this in a future article.

fennel

one way to get the benefits of love2d without the drawbacks of lua is to use a language that compiles to lua! fennel is my favorite of these. fennel is a lisp, which means it might be very different from how you’re used to programming video games. if you can get used to it, it has many delightful benefits that are beyond the scope of this article to enumerate. give it a a shot, see if you like it.

moonscript

moonscript is another language that compiles to lua. it adds a lot of features to lua, like a proper class-based object model, and a bunch of ruby-style syntactic sugar. if that sort of thing appeals to you, you might like moonscript. also, it was made by the guy who made itch.io.

typescript to lua

typescript is yet another language that compiles to lua! i forgot about it in the original verison of this post, but i’ve actually used it before. it’s good! it fixes the biggest problem with lua, which is its lack of static typing. to many beginner programmers, static typing can feel unecessarily onerous, but the truth is it catches so many obnoxious bugs at compile time that would otherwise be super annoying to track down. if i were to seriously make a project with love2d, i’d make it with typescript to lua.

g3d

g3d is a framework which is designed to be a simple interface for love2d’s 3d capabilities. it’s great! it makes loading 3d models and textures easy, and provides built-in capabilities for things like first person movement. g3d is what really pushes love2d over the edge into being the perfect unity replacement for most projects.

c/c++

like c#, c and c++ really need no introduction. c is one of the oldest programming languages still in common use, and c++ has been the dominant language of video game development for almost 30 years. c++ is the language i’m using for my side projects right now!

the way i am using c++ is rather strange. my preferred way to interface with c++ is to use as little of its features as possible, and basically just write c, but using c++ features when they’re useful. the main benefit of c++ for me is that the flecs API is much simpler in c++ than in c. other than that, i just write c. i ignore c++’s object-oriented features, its templating, and all the other weird crap they’ve added completely.

the main advantage of c and c++ is also its main disadvantage: manual memory management. people like me who understand pointers will often say that they’re simple, and people who don’t understand pointers often view them like they’re impossible to understand. as someone who came to understand pointers fairly recently, what i will say is that i don’t know how it happened. it just sort of clicked at a certain point and i stopped getting so many segfaults.

the main disadvantage of c’s approach to manual memory management is that several API choices make it very easy to accidentally create memory leaks or undefined behavior. the good news is that modern compilers are often able to identify potential issues and warn you about them. i am personally intrigued by zig as a potential c replacement, though it hasn’t been around long enough for me to feel confident recommending it.

the main advantage of manual memory management is performance. garbage collection is computationally very expensive, and in real-time computing applications like games, you wind up having to program in a way that puts as little pressure on the garbage collector as possible if you want your game to run at a decent framerate. if you don’t care about performance, or you think that “memory is cheap,” you’re wrong! many, many people cannot afford high-end hardware to run video games on, and they deserve to be able to play your games. we talk so much about “accessibility” in terms of options menus, but never in terms of who can literally afford to experience the games we make.

furthermore, the constant churn of ever-improving computer hardware is not good. it creates a lot of e-waste, which generally gets dumped on the poorest countries with the least ability to deal with the consequences. also, manufacturing hardware is a resource-intensive process. it uses a lot of water, releases a lot of greenhouse gases, and involves the mining of a lot of rare earth metals, some of which are conflict minerals.

programming in c is not the solution to any of these problems, but it’s something to think about as you program your game and select your tools.

SDL

SDL is the gold standard for video game graphics APIs. some of the tools we’ve already discussed are based on SDL. SDL is a utility for handling window creation, simple 2D drawing, and creating and managing opengl or vulkan contexts. it does very little for you. if your game is very simple and 2D, you might be able to get away with using just SDL. if you’re making something more complex, or anything 3D, you may want to step up a bit.

SDL’s main utility is as a platform layer. it abstracts away a lot of the details of individual operating systems, so you can easily target multiple platforms without having to write a bunch of bespoke code. this is why a lot of other frameworks are built on top of it – both love2d and FNA are based on SDL.

raylib

i have not personally used raylib, but i plan to try it out in the future. it seems rad. it’s essentially XNA but for c. it has lots of builtin tools to make 3d game programming easier. it also has a very active community and lots of examples to learn from.

flecs

as i hope i’ve made clear, if you’re going to be programming a game in 2022, you should be using an ecs. and the best ecs for C is flecs. lots of features, a great api, super fast, and it even has a cool dashboard to allow you to profile and debug the running ecs. i wish it had the messaging features of moontools.ecs, but other than that it’s great.

rust

rust is an exciting and relatively new programming language that has a lot of potential. unfortunately, i just don’t think the ecosystem is quite there yet, and nobody has yet shipped a cross-platform game with it. it’s something to keep an eye on though.

the main benefits of rust, hypothetically, is that it uses extensive compile-time checks to allow you to write code that’s as fast as c but without fear of memory leaks or undefined behavior. exactly how it does this is beyond the scope of this article.

the main disdadvantage is that the way it forces you to program to provide these guarantees of memory safety is rather unnatural for people coming from most programming languages. if you have a background in haskell or ocaml, rust might come very naturally to you, but for the rest of us, it requires substantial rewiring of the brain. until you’ve achived that rewiring, it comes with a lot of friction.

i have written two large projects in rust: a web server for a streaming platform, and a media player. my experiences led me to conclude that it wasn’t quite ready for video game applications yet. mainly, i think that rust’s memory safety guarantees are extremely important for any software that’s going to be connected to the internet, and its drawbacks are less important than writing software that is better at prevending problems like heartbleed. we saw fairly recently that dark souls 3 had a remote code execution bug that might had been prevented if the engine were written in rust. in the context of a single-player video game, though, memory safety issues aren’t really a problem.

bevy

of the options available for making games in rust, bevy is the only one i would recommend. it comes with a fully-featured ECS, and tools for rendering both 2D and 3D games. ultimately, i am skeptical about it in some of the ways i am skeptical of godot. it has too many features. what rust really needs is a good, well maintained tool with an api like raylib, FNA, or love2d. bevy is not that, and as far as i can tell, there isn’t something like that available for rust, though some people are trying.

intentional omissions

some things have been omitted from this list on purpose. if the thing you want to pitch me on is in this section, please do not talk to me about it. i don’t care.

SFML

SFML is a library that is slightly higher level than SDL. it’s more “batteries included” than SDL. many people claim it is more “beginner friendly,” than SDL, though i think this is a stupid metric by which to evaluate the quality of tools. SDL breaks up some of its functionality into component libraries, while SFML has it all built in.

the biggest difference is that SFML is built in C++ and SDL is built in C. this is somewhat restrictive. C++ is not a good programming language, and i recommend using its features as little as possible.

sfml’s API is, unfortunately, designed in a rather object-oriented way that makes programming clunky. for instance, sprite objects, rather than just storing a texture for you to specify the on-screen location of later in the draw call, store transform and color data alongside them. this makes programming harder, because now you have to synchronize between the location of your game entity, and of this sprite entity. or, even worse, you have to use inheritance, which opens up an entire mess of problems.

overall, i think SDL is a better option for simple rendering, and raylib is a better choice for more complex rendering.

javascript

javascript sucks, plain and simple. people will try to tell you it’s gotten better, it has not. javscript’s var keyword featured some bizarre scoping behavior that they had to fix by adding new ways to declare variables, and if you accidentally use var instead of let or const you’re hoisted. javascript’s weak type system leads to some absolutely baffling type coercions, which led to the creation of the === comparator. forget to use that extra equals sign, and you’re hoisted.

javascript shares all the problems of lua, but with infinitely more bizarre edge cases, and even worse performance because heavy lifting can’t be passed off to C code.

the programming language zig, in a document outlining why you should use zig and not rust, D, or C++, says something i think is illuminating:

C++, Rust, and D have such a large number of features that they can be distracting from the actual meaning of the application you are working on. One finds oneself debugging one’s knowledge of the programming language instead of debugging the application itself.

javascript is, even more than those languages, the peak of this kind of language design. javascript gets in the way. at every turn in its design, the worst possible decision was made. is that because it was designed in extreme haste by moronic homophobe brendan eich? it’s impossible to say.

it is possible to use javascript to compile desktop applications, with tools like electron and nw.js. this generally results in terrible performance, as anyone who’s used a desktop application developed in the last five years can tell you. this is because these tools deploy entire web browser just to run a single application, which is insane. it is, of course, impossible to port a game written in javascript to consoles.

many people use javascript to make games for the browser. you should not do this. the browser is where software goes to die. in my lifetime, countless software artworks have been lost to time because browser manufacturers constantly make changes that break compatibility. making software for the browser is a hell nightmare of compatibility issues, which become more and more salient the more complex your program is. maintaining a browser program is a similar nightmare, in which you are jerked around by the whims of google engineers who have never considered that there are people in this world whose whole job is not web development who might want to make web software.

there is nothing to gain in a game development context from using javascript that you can’t get from using one of the languages discussed above. if you desperately need dynamic typing, use lua, or fennel, or moonscript, or sign up for the luxe beta. if you want to deploy to the browser, use haxe, which will also let you deploy native apps for the desktop that aren’t smuggled inside of a web browser.

javascript is not a language for beginners. it is a wasteland of bizarre bugs and inconsistent behavior that will confound and infuriate anyone who has not spent years getting to know its idiosyncrasies. i would not wish javascript on my worst enemy.

conclusion: killer whales, or: “DIY or die”

i don’t know how the cool, hip, punk way to make video games became relying on proprietary software made by megacorporations. i don’t know how people who consider themselves “leftists” justify mocking and deriding those who would challenge the dominance of those megacorporations. i don’t know how we’ve all deluded ourselves into seeing this megacorporate dominance as “democratization.”

what i do know is: it doesn’t have to be like this. making video games doesn’t have to be as miserable as it is right now. it will never be easy, but we are making it harder because corporations have convinced us that we are incapable of doing anything for ourselves. i don’t think you shoud make your own engine because i’m an elitist: on the contrary. i believe in you. i believe that when big corporations tell you that making games is too hard for your puny little brain and you should just use their tools that make it easy, they are lying to you to make money.

fuck unity. fuck epic games. fuck yoyo games (especially fuck yoyo games.) fuck all the predatory snake oil salesmen capitalizing on people’s desire to make art.

in the very near future, i will be publishing tutorials on this blog about how to make video games with some of the tools outlined above. the first one is going to be about making games with love2d.